Preventing Overfitting in Decision Trees

Question 1

Screenshot taken from Coursera

Question 2

Screenshot taken from Coursera

Question 3

Screenshot taken from Coursera

Question 4

Screenshot taken from Coursera

Question 5

Screenshot taken from Coursera

Question 6

Screenshot taken from Coursera

Question 7

Screenshot taken from Coursera

Question 8

Screenshot taken from Coursera

Answer

At this node, the y values have:

3 True (+1)
1 False (-1)

The classification error is: 1/4 = 0.25

Question 9

Screenshot taken from Coursera

Answer

x1 has 2 zeros and 2 ones, so spliting by x1, we have the tree structure as follow



In [47]:

    
print '                        %s' % 'root (3, 1)'
print '         |---------------|----------------|'
print '         |                                |'
print '         |                                |'
print '         |                                |'
print '      node_1 (2,0)                  node_1 (1,1) '









    



                        root (3, 1)
         |---------------|----------------|
         |                                |
         |                                |
         |                                |
      node_1 (2,0)                  node_1 (1,1)

Classification error = 1/4 = 0.25

Question 10

Screenshot taken from Coursera

Answer

x2 has 1 zeros and 3 ones, so spliting by x2, we have the tree structure as follow



In [48]:

    
print '                        %s' % 'root (3, 1)'
print '         |---------------|----------------|'
print '         |                                |'
print '         |                                |'
print '         |                                |'
print '      node_1 (2,1)                  node_1 (1,0) '









    



                        root (3, 1)
         |---------------|----------------|
         |                                |
         |                                |
         |                                |
      node_1 (2,1)                  node_1 (1,0)

Classification error = 1/4 = 0.25

Question 11

Screenshot taken from Coursera